perf: Add composite index on jobs table for efficient job fetching#1364
Merged
MilagrosMarin merged 1 commit intomasterfrom Jan 30, 2026
Merged
perf: Add composite index on jobs table for efficient job fetching#1364MilagrosMarin merged 1 commit intomasterfrom
MilagrosMarin merged 1 commit intomasterfrom
Conversation
Add INDEX (status, priority, scheduled_time) to optimize the critical populate query: SELECT ... FROM jobs WHERE status='pending' AND scheduled_time <= NOW() ORDER BY priority ASC, scheduled_time ASC This index covers the WHERE clause and ORDER BY, eliminating table scans and filesorts for large job queues. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
MilagrosMarin
approved these changes
Jan 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a composite index to the jobs table to optimize the critical job fetching query used during
populate(reserve_jobs=True).Problem
When populating with job reservation, the following query runs frequently:
Without an index, this requires a full table scan and filesort, which becomes slow with large job queues (thousands of jobs).
Solution
Add a composite index that covers both the WHERE clause and ORDER BY:
This enables:
status='pending'scheduled_timewithin the indexpriority, scheduled_time(no filesort needed)Testing
test_jobs.pytests passSHOW INDEX🤖 Generated with Claude Code